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The accountability 
framework that has 
been developed 
through the Bologna 
Process is worth 
learning from and 
thinking deeply about 


Accountability 

“Light” 

Our Version Is 
Going the Way of the 
Dollar vs. the Euro 


It WAS July 14, 2006, when the Spellings Commission on the 
Future of Higher Education posted the second draft of its final re- 
port on the Web for all to see. It was the first of 
the commission’s drafts in which the alleged illit- 
eracy of U.S. college graduates was placed in bold space along 
with all the other flagellations of higher education in which the 
commission delighted. The text of the Web posting, though, was 
an unhappy validation of the complaint: the cases of violation of 
subject-verb agreement leaped off the electronic pages, along with 
paragraphs that can be kindly described as strings of non-sequiturs. 
The commission’s draft would not be awarded a C- in English 
Composition on a generous grading day. Yet no one noticed the 
irony. After all, this was a distinguished panel. Higher education 
had not seen a federal undertaking such as this since 1984, and it 
was getting visible attention in the general press for topics other 
than college costs. 

But it would not have been hard to notice, in both the Web- 
posted drafts and the final report of the commission, that the only 
references to higher education outside the United States recited 
ill-informed data putatively demonstrating how far we had “fallen 
behind” other nations. 1 Nobody on the commission or its staff 
had bothered to examine the ways other countries are dealing 
with issues we also face — access, degree completion, and, most of 
all, accountability. In its insular and chauvinistic way, this major 
pronouncement on, and recommendation of remedies for, the state 
of U.S. higher education pretended, without looking, that the world 
is learning more, and we are not — a conclusion designed more to fit 
our appetite for bad news and self-degradation than to enlighten. 

CLIFFORD ADELMAN has been senior associate at the Institute for Higher 
Education Policy since the fall of 2006. Prior to that, he served for 
twenty ^even years as senior research analyst at the U.S. Department of 
Education, and ten years in both teaching and academic administration. 
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Now, if the rest of the world is learning more, 
would that be surprising? And in a global econ- 
omy, would it really be so disappointing? Cul- 
ture and language ensure that the world is not 
flat, but in tenns of knowledge it is, and the 
world’s knowledge content, like the oceans, is 
rising. To the extent that the world learns more, 
we all benefit, and we cannot pretend that the 
United States holds a monopoly on knowledge. 
More seriously, because credentials awarded are 
used as a proxy for learning in international — as 
well as domestic — comparisons, nobody knows 
for sure whether students in other countries are, 
in fact, learning more. But nowhere in recent 
years have academic leaders, faculty, and stu- 
dents themselves wrestled more with the knots 
of credentials and learning than in the old na- 
tions of Europe. Through the largest recon- 
struction of higher education ever undertaken 
anywhere — known as the Bologna Process — 
they have come up with a convincing and cred- 
ible scaffolding of accountability. When one 
considers what the Europeans have done — and 
where they have more to do — one realizes that 
we in the United States have a long way to go 
in understanding what “accountability” means. 

The Bologna Process 

The Bologna Process is named for the city where, 
in 1999, the education ministers of twenty- 
nine countries first reached a meeting of the 
minds on the future of higher education in 
Europe. It is an inevitable outgrowth of the 
process of European integration that began in 
1950. By the 1990s, and certainly after the 
Maastricht Treaty of 1992, Europe began to re- 
semble a quasi-federal arrangement. Although 
there were few economic borders, the common 
European workforce was, ironically, stuck 
within political borders because individual 
countries did not yet fully recognize — or even 
understand — their neighbors’ education cre- 
dentials. In order to allow for the recognition 
of higher education credentials across borders 
and, thus, to provide mobility for the advanced 
knowledge workforce, some convergence of ed- 
ucational practices and standards was needed. 

Bologna became a force for converting edu- 
cation systems into similar forms and processes. 
Its discourse is a type of technology transfer that 
brings nations from different platforms of edu- 
cational development to the point of embracing 
compatible paradigms. In other words, they 
wind up singing in the same key — though not 
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the same tune. With its simul- 
taneous respect for distinctive 
national traditions and institu- 
tional autonomy, the Bologna 
Process became doubly attrac- 
tive as the only game in town. 

In 1999 and subsequently — 
forty-six countries are now in- 
volved — the ministers agreed to an action 
agenda for dissolving educational borders in 
the same way that economic borders had been 
dissolved. The agreement was an inevitable 
consequence of reforms that had been stirring 
across European education during the 1990s. 
The feeling was that, having lost their way 
and their world leadership, ancient systems of 
higher education needed a kick-start. In 1998, 
the ministers of the four largest countries in Eu- 
rope — France, Germany, Italy, and the United 
Kingdom — met at the Sorbonne in Paris and 
planted the seeds from which the European 
Higher Education Area has grown. The 
Bologna ministers optimistically set 2010 as 
the date by which the transformations they 
imagined were to be realized. But inevitable 
inertia and resistance at the institutional level, 
new provisions, and additional partners have 
rendered the 2010 completion marker a mirage; 
2020 is more likely. 

The Bologna Process is a huge undertaking. 
Its action lines include revising degree cycles, 
establishing degree qualifications frameworks, 
overhauling the credit system, increasing access 
via more flexible paths into and through higher 
education, validating student attainment, es- 
tablishing a quality assurance system, and in- 
creasing the attractiveness of the European 
Higher Education Area for foreign students. 
Some four thousand institutions, which enroll 
sixteen million students, are now participating. 
Some of what has been accomplished has al- 
ready been adapted in Latin America, North 
Africa, and Australia. Indeed, the Bologna 
model is well on the way to becoming the dom- 
inant global paradigm for higher education by 
roughly 2025. We cannot afford to ignore it. 

Real accountability 

The Spellings Commission report and the re- 
sponse to it from within the U.S. higher edu- 
cation community give the impression that 
accountability is a very simple matter. In order 
for colleges and universities to be “accountable” 
to those who subsidize them or pay their tuition 


and fees, they need only to 
make public their graduation 
rates, demographic mix, job 
placement rates, and some 
data from the N ational Survey 
of Student Engagement on 
what their students like about 
opportunities and services. 
They may perhaps also throw in a test or two 
to show that a sample of their students know 
how to write or solve a problem or that a ran- 
dom set of one hundred student volunteers 
produced an X-standard deviation improve- 
ment between entrance and graduation on a 
test of “critical thinking” (which improve- 
ment, of course, is attributed entirely to the 
institution 2 ). But this is show, not substance. 
Wish lists for student learning, flowery goals 
statements, and agendas of commitment for 
the twenty-first century are sometimes offered 
in parallel rubrics and posed as accountability 
statements. But none of these efforts explains 
what credentials represent or what students 
must do to earn them. There are no public ref- 
erence points or performance criteria. Stu- 
dents — our constituents who ought to be our 
partners as well — played no role in fashioning 
these efforts. And given how divorced these 
various approaches are from the actual work 
of most faculty and their students, their likely 
impact on student learning is limited indeed. 

So what are the Europeans doing differently? 
They have essentially created an accountability 
loop that is executed by national systems and 
institutions. Although still a work in progress, 
the European accountability loop begins with 
degree qualifications frameworks and circles 
back with a document that serves as a public 
warranty, attesting to the institution’s judg- 
ment and the student’s achievement. In be- 
tween are credential qualifications frameworks 
at the disciplinary level and a student-centered 
credit system to which a growing number of 
institutions have added levels of challenge. 
Countries outside of Europe have already 
adapted some of these elements for use within 
their national systems or institutions, and it is 
worthwhile to consider whether some elements 
might usefully be adapted to higher education 
in the United States. 

Qualifications frameworks 

Within the context of the Bologna Process, a 
qualifications framework is a statement of the 


It is worthwhile to 
consider whether 
some elements might 
usefully be adapted to 
higher education 
in the United States 


O 

a. 

o 

t- 

a 

u 

0£ 


< 

Ul 

u. 


Fall 2008 Liberal Education 9 


o 

CL 

o 

o 

u 

oc 


< 

lu 


learning outcomes and competencies that a 
student must demonstrate in order to be 
awarded a degree at a specific level. It is not 
simply a statement of objectives or goals; it is 
not a wish list. Rather, it is a set of performance 
criteria. When an institution of higher educa- 
tion is governed by a qualifications framework, 
it must “demonstrate” that its students have 
“demonstrated ” — all of its students, not merely 
a hundred volunteers who take a standardized 
test. While a qualifications framework does 
not dictate how that demonstration takes 
place or the nature or form of assessments 
employed, it does provide learning outcome 
constructs within which the demonstration 
is conducted. 

A second key characteristic of a qualifica- 
tions framework is that it clearly indicates the 
criteria for each level of credential offered. 
The language of the framework accomplishes 
this through a “ratcheting up” of learning 
benchmarks — that is, by increasing the level 
of challenge at each rung on the degree lad- 
der. This “ratchet principle” pervades all of 
the content challenge and performance state- 
ments of Bologna, from individual courses to 
degrees, and has penetrated the credit system 
as well. This principle embodies content and 
performance standards, and it functions as an 
engine of accountability. 

There are three levels and types of qualifica- 
tions frameworks under Bologna: transnational, 
national, and disciplinary/field. The trans- 
national Framework of Qualifications of the 
European Higher Education Area provides a 
wide-angle set of generic markers that clearly 
distinguish among the “short-cycle” degree 
(the equivalent of our associate’s degree), the 
bachelor’s degree, and the master’s degree. The 
distinctions are drawn not in terms of a num- 
ber of credits or a minimum grade point aver- 
age, but rather in terms of student learning 
outcomes across five bands of requirements: 
knowledge and understanding; application of 
knowledge and understanding; fluency in the 
use of increasingly complex data and informa- 
tion; breadth, depth, and range of audiences 
for communication; and degree of autonomy 
gained for subsequent learning. The language 
used to describe these requirements “ratchets 
up” the level of challenge as one moves up 
the degree ladder. This framework has been 
accepted by all forty-six countries participating 
in the Bologna Process. 


One might expect each national higher 
education system to develop its own compati- 
ble version of the Qualifications Framework 
for the European Higher Education Area — 
adding detail, taking into account the pecu- 
liar varieties of its own institutions as well as 
their historical missions and commitments, 
and perhaps including “intermediate” qualifi- 
cations between the three degrees. In prac- 
tice, however, that’s not how the national 
qualifications frameworks have developed. 
Creating consensus on a national framework 
is a time-consuming challenge, and to date, 
only seven of the forty-six participating coun- 
tries have completed the task. Among them 
are five highly distinct models that illustrate 
the way the Bologna countries can achieve 
“convergence” with variation. The Irish and 
Scottish framework is comprehensive, from 
kindergarten through the doctoral level, 
while the German framework offers a more 
parsimonious articulation of how university 
students must demonstrate knowledge through 
what are called instrumental competencies, 
systemic competencies, and communicative 
competencies. The Dutch framework refer- 
ences labor market positions and tasks, and 
the Swedish framework reaches into nineteen 
specific applied disciplines that lead to licensure 
occupations (e.g., nursing). The French frame- 
work requires every degree program in the 
country to undergo a central registry review, 
and the program dossier is made public via 
the Internet. 

Tuning and benchmarking 

The disciplinary qualifications frameworks are 
developed through what is called the “tuning” 
process. “Tuning” is a methodology for pro- 
ducing “reference points” that faculty can use 
in developing statements of learning outcomes, 
levels of learning, and desired competencies 
in the disciplines. The purpose is to ensure 
that these statements are both transparent 
and comparable. This does not mean that the 
content is standardized, however. The crite- 
rion-referenced competency statements that 
result from the tuning process are not straight- 
jackets. To reprise the music metaphor: every- 
body winds up with the same music staves, 
range of time signatures, tempo commands, 
and system of notation. Then, all programs in 
the same discipline sing in the same key — en- 
gineering in A-minor, business in B-flat — but 
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they don’t necessarily sing the 
same tune. The first phase of 
the tuning project involved 
nine fields in 138 institutions 
across sixteen countries. In 
the second phase, sixteen 
other fields joined the model. 

Tuning is the largest Bologna “export” to date: 
twelve disciplines in 182 universities in eigh- 
teen Latin American countries have adopted 
the methodology. 

However attractive “tuning” may be, it is 
not always executed to match its purpose. 
Faculty members have difficulty writing crite- 
rion-referenced learning outcome statements, 
and that’s something the Europeans — and we 
in the United States — need to work on. Disci- 
pline-based benchmarking, a strong suit of the 
United Kingdom’s Quality Assurance Agency 
(QAA), is an especially prominent alternative. 
Like tuning, benchmarking provides reference 
points and boundaries for designing, modify- 
ing, and evaluating the presentation of a disci- 
pline within an institution or a group of 
similar institutions. The QAA benchmarking 
statements for degrees in accounting, for ex- 
ample, clearly describe the subject-specific 
knowledge, skills, and cognitive operations 
that the graduate will have “demonstrated” at 
a level crossing a threshold standard. While 
institutions choose their own forms of assess- 
ment and set their own thresholds, a student 
who does not “demonstrate” does not earn 
a degree. 

So what can we learn from the European 
experience with qualifications frameworks, 
tuning, and benchmarking? Accountability 
begins with the establishment of public defin- 
itions of degrees and criterion-referenced 
statements of performance. Once such defini- 
tions are established, the credential-awarding 
institution can say, “this is what this student 
did, and this is what the degree represents.” 

The importance of discipline-specific con- 
tent has been a consistent theme of the 
Bologna Process. After all, students go to col- 
lege to earn a degree in anthropology, me- 
chanical engineering, or nursing; they go to 
community colleges to earn a degree in med- 
ical technology or commercial art. When you 
ask them what they are studying, they will an- 
swer by telling you about their discipline or 
field. And, of course, faculties the world over 
are also organized by discipline or field. When 


our students enter a major — 
and when they graduate — 
they deserve to know what they 
are in for, and, later, what they 
have accomplished. Students 
are not likely to get this clarity 
from the institutional gradua- 
tion rates, indicators of student engagement, 
“value-added” metrics of “critical thinking,” 
and generalized curricular goals one finds in so 
many “accountability” pronouncements in 
the United States. Our students deserve better. 

Euro-Credits 

While the European Credit Transfer System 
(ECTS) is built on a fundamentally different 
assumption from that used for U.S. higher ed- 
ucation credits, if it plays out in its ideal form, 
it will go beyond functioning as a measure of 
accumulation and become another account- 
ability tool. 

In the United States, credit assignments 
are based on faculty contact hours, with the 
assumption that, in relation to each faculty 
contact hour, the student engages in other 
types of learning activities. In contrast, ECTS 
uses the student as the primary reference point, 
asks how many hours the average student must 
spend to accomplish the various tasks in a 
course module, and converts the total to credits. 
The conversion formula is based on the length 
of the academic year, the total number of hours 
available for study during that year, and a refer- 
ence marker of sixty credits per academic year. 
The conversion formula ranges from twenty- 
five to thirty workload hours per credit. 

The virtues and drawbacks of such a system 
are obvious. When faculty really have to 
think about what they are asking students to 
do, there is strong potential for curricular 
change. Redundancies, gaps, and opportuni- 
ties for cyber-efficiencies can be identified, 
and appropriate changes in coverage and de- 
livery can be made. The Bologna literature 
describes such reforms in fields ranging from 
music to chemistry. The obvious drawback can 
be simply phrased: when you have a formula, 
it becomes the default. Water finds the easiest 
way to flow downhill, and both academic 
administrators and faculty become much too 
mechanical and downright sloppy in assigning 
credits to course modules. 

How does one connect student workload 
to learning outcomes, to the principle of the 
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ratcheting up of challenge and, hence, to 
qualifications frameworks and the structure of 
accountability? We don’t worry about such 
things in the United States. We grant three 
credits for Introduction to Sports and three 
credits for Econometrics or assign nine credits 
for a Recreation Practicum and four credits for 
Neuropsychology — and brush such obvious 
dissonances under the rug. Nobody — not the 
Spellings Commission or the various “ac- 
countability”'Oriented responses of the U.S. 
higher education community — has evidenced 
any desire to acknowledge, let alone repair, 
these obvious holes in the academic integrity 
of our credit system. In the United States, 
credits live in the office of the vice president 
for finance, not the vice president for academic 
affairs; the student is incidental to it all. If we 
care about accountability for student learning, 
and if we are serious about matching practice 
to rhetoric, then we may need a redesign. If 
ECTS plays out in ideal form even among a 
plurality of Bologna participants, it will show 
us the way. 

Given different modes of student work in the 
disciplines, our European colleagues have gone 
about the task of linking workload to learning 
outcomes with alternative proxies. In the most 
intriguing of these approaches, that taken by 
the United Kingdom and Scotland, credits are 
placed within degrees of challenge. That link — 
between the measure of estimated student 
time-on-tasks and the level of demand inherent 
in those tasks — creates a “credit level” defined 
as “an indicator of the relative demand, com- 
plexity and depth of learning and of learner 
autonomy.” 3 There are nine such credit levels, 
and each carries a generic description that is 
independent of discipline but applicable to all 
disciplines. Every course is tagged with a credit 
level, and the number of credits awarded is 
treated as a separate issue. Once levels such as 
these are established, degree qualifications can 
be set in temis of minimums — 40 percent of 
credits at level six, for example, and 65 percent 
of credits at levels five and six. If all colleges 
and universities in the United States added 
such indicators of the challenge of content to 
credits, quarrels about credit transfer would 
diminish considerably. 

Diploma Supplements 

After qualification frameworks, tuning, and 
an ideal credit system marked with parallel 


structures of challenge, what is left in this very 
different scaffolding of accountability? What 
evidence of learning and attainment does the 
graduate carry forward into the world, and 
how is that evidence communicated for all to 
see? The piece of paper we call a diploma does 
not say much, and our European colleagues 
have added something very important to it. 

As the final element in the Bologna ac- 
countability loop, the Diploma Supplement 
serves as a kind of warranty, providing evidence 
of the graduate’s learning and attainment. In 
addition to including a transcript and indicat- 
ing superior performance (i.e., honors), the 
Diploma Supplement is intended to convey 
information about 

• the national system in which the degree 
was awarded and the position of the degree 
within that system’s credentialing hierarchy; 

• the status, type, and accreditation of the 
institution awarding the degree; 

• the purpose and function of the credential; 

• the student’s major field of study as well as 
the duration and the entry requirements of 
the program in which the credential was 
granted; 

• discipline-level qualifications and degree 
requirements, including internships, theses, 
and final projects; 

• modes of study, including enrollment inten- 
sity and distance learning. 4 

Even if they included only the information 
listed above, Diploma Supplements would go 
some way toward providing public assurance 
about higher education credentials. Some 
serious revisions are needed, however. The 
absence of discipline-level qualifications state- 
ments, for example, is a very unhappy omis- 
sion in light of the qualifications frameworks 
core of Bologna. And the intended warrantee, 
the student, plays too minor a role in the cur- 
rent version of the Diploma Supplement for 
the loop to close. 

In addition to providing a public warranty 
for our students, the creation of a U.S. version 
of the Diploma Supplement would put institu- 
tions of higher education on the public record 
in terms of their standards for degree qualifica- 
tions, and it would hold them to consistency. 
Setting aside the transcript as a separate docu- 
ment (as it should be), what should our version 
of the Diploma Supplement contain in addi- 
tion to the bullets above covering the institu- 
tion, the degree and its function, modes of 
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study, and “compressed signals” of superior aca- 
demic performance? I advocate the following: 

• Not-so-standard boilerplate indicating all 
other institutions attended by the student 
from which credits were accepted as well as 
the percentage of the student’s credits that 
were earned at the institution awarding the 
degree 

• A tuning-type qualifications statement for 
the student’s major and information about 
any required internships, theses, final pro- 
jects, portfolios, or comprehensive exami- 
nations 

• The title and a short description of the stu- 
dent’s thesis or final degree-qualifying 
project, if applicable 

• Any external certification examinations 
passed or licenses granted to the student 
(although the institution is not the award- 
ing body in these cases, the institution 
certifies that it has recognized and recorded 
them) 

• A maximum of two noteworthy and docu- 
mented services performed by the student for 
the institution or its surrounding community 

• Student research, creative, or service par- 
ticipation, if applicable (field, title of pro- 
ject, and faculty sponsor) 

• Documented proficiency in languages other 
than English, including method of docu- 
mentation 

To be effective and credible, these markers on 
a Diploma Supplement should be limited, 
based on unobtrusive institutional records of 
the student’s activities, concentrated on 
achievements related to the degree awarded, 
and verifiable and validated by whoever signs 
the document on behalf of the institution. 
Otherwise, they are properly part of a resume. 

Such a Diploma Supplement would close 
the accountability loop. To repeat: it is both a 
public warranty and a private assurance of the 
meaning of the degree, of the standards for 
awarding it, and of what the student did to earn 
it. Thus the loop returns to the public defini- 
tion of degrees and public criterion-referenced 
statements of performance, with real students 
carrying the evidence of having met both. This 
would be a far more convincing enactment of 
accountability than what we have seen in the 
United States over the past two years. 

The accountability framework that has 
been developed through the Bologna Process 
is worth learning from and thinking deeply 


about, and I urge academics in the United 
States to take it very seriously — before the 
“dollar” of their enterprise loses further value, 
no matter how many degrees they award. □ 

To respond to this article, e-mail liberaled@aacu.org, 
with the author’s name on the subject line. 


NOTES 

1. Comparisons among national systems rely princi- 
pally on the methodology of the Organization for 
Economic Co-operation and Development (OECD) 
and its highly visible vehicle, Education at a Glance. 
Yet OECD itself has for some time been unhappy 
with its higher education indicators and, in the re- 
cently released 2008 edition of Education at a 
Glance, has made the first of a series of changes in 
data reporting: an indicator based on cohort gradu- 
ation rates for first-time, full-time students in bach- 
elor’s degree-granting institutions has been added. 
OECD’s traditional indicator for postsecondary at- 
tainment relies on population ratios — i.e., from 
census data or labor force surveys in each country, 
the number of people holding postsecondary degrees 
in relation to the total working-age population. 
Variations on this indicator also use population 
ratios for specific age brackets (e.g., twenty-five to 
thirty-four). The problems with this approach are 
numerous, beginning with different census 
methodologies and different definitions of degrees. 
This article is not the occasion for elaboration. For 
an overview, I recommend Jane Wellman’s Apples 
and Oranges in the Flat World: A Layperson’s Guide 
to International Comparisons of Postsecondary Educa- 
tion (American Council on Education, 2007). I 
also recommend reading the technical notes in Ed- 
ucation at a Glance — particularly for the new indi- 
cator — as the notes demonstrate national variances 
in definitions, inclusions, and exclusions. 

2. Not considered are a host of potential intervening 
variables such as whether the student volunteers 
were transfers- in or attended other schools as un- 
dergraduates, whether they were parents and/or 
veterans, whether they held jobs that provided 
learning challenges, whether they practiced debate 
in Bible or Talmud study in religious institutions, 
and whether their significant others spoke languages 
other than English. 

3. Credit and HE Qualifications (Joint Credit Bodies 
for England, Wales, and Northern Ireland, 2001 ). 
For text of a sample of four of these credit levels, 
see The Bologna Club: What U.S. Higher Education 
Can Learn from a Decade of European Reconstruction 
(Institute for Higher Education Policy, 2008), 60. 

4. The official “Outline Structure for the Diploma 
Supplement” is available online at www.ec.europa.eu/ 
education/policies/rec_qual/recognition/ds_en.pdf. 
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